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Abstract. Let Gm be a real reductive Lie group acting on a manifold M. 
M. Kashiwara and W. Schmid in IKaSchrnI constructed representations of Gr 
using sheaves and quasi-G|g-equivariant I'-modules on M. In this article we 
prove an integral character formula for these representations (Theorem^. Our 
main tools will be the integral localization formula recently proved in IL3I and 



the integral character formula proved by W. Schmid and K. Vilonen in ISchV2l 
(originally established by W. Rossmann in IRoH in the important special case 
when the manifold M is the flag variety of C (^E 0E - the complexified Lie al- 
gebra of Gr . In the special case when Gr is commutative and the IJ-module is 
the sheaf of sections of a Gig-equivariant line bundle over M this integral char- 
acter formula will reduce to the classical Riemann-Roch-Hirzebruch formula. 
As an illustration we give a concrete example on the enhanced flag variety. 
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1. Introduction 



We consider pairs of Lie groups: a real linear reductive group G] 
its complexification Gc- For example: 

GL(n,R) C GL{n,C) S'i(n,M) 
GL+(n,M) C GL{n,C) SO{n,C) 
U{n) C GL(n,C) SU{n) 



sitting inside 



C 

c 
c 



SL{n,C) 

SL{n, C) M) C Sp{n, C). 

SL{n,C) 



Suppose that Gc acts algebraically on a smooth complex projective variety M . Fix 
a Gc-invariant open algebraic subset U C M , and take a Gc-equivariant algebraic 
line bundle (E, Ve) over U with a Gc-invariant algebraic flat connection Ve- Let 
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Sr C M be an open GR-invariant subset (which may or may not be Gc-invariant) 
and consider the cohomology spaces 

(1) HP{Sm,0{E)), pez. 

The classical Riemann-Roch-Hirzebruch formula computes the index of E, i.e. 
the alternating sum 

^(-If dim7?P(5H,0(E)) 

p 

with Str = U = M. For general and U, however, these dimensions can be infinite. 
To work around this problem we regard the vector spaces as representations of 
Gr, and, as a substitute for the index, we ask for the character of the virtual 
representation 

^(-lfi7P(5M,0(E)). 

p 

(Recall that for finite-dimensional representations the value of the character at the 
identity element e G G equals the dimension of the representation.) 

In this article we establish an integral formula for characters of virtual repre- 
sentations of this kind in a more general setting of I?-modules. Existence of such 
integral character formula was conjectured by W. Schmid. My work toward this 
formula has led to the integral localization formula proved in jL3| . 

We use the following convention: whenever ^ is a subset of B, we denote the 
inclusion map A ^ B by Ja^b- 

2. Preliminary results 

Let Gc be a connected complex algebraic reductive group which is defined over 
R. We will be primarily interested in representations of a subgroup Gr of Gc 
lying between the group of real points Gc(IR) and the identity component Gc(K)°. 
We regard Gr as a real reductive Lie group. Let gc and 0r be the respective Lie 
algebras of Gc and Gr. Let B denote the flag variety of gc- It is a smooth complex 
projective variety consisting of all Borel subalgebras of gc- The group Gc acts on 
B transitively. 

If the group Gr is compact, then all irreducible representations of Gr can be 
enumerated by their highest weights A lying in the weight lattice A C ig^. The 
Borel- Weil-Bott theorem can be regarded as an explicit construction of a holomor- 
phic Gg-equivariant line bundle C\ ^ B such that the resulting representation of 
Gr in the cohomology groups is: 

HP{B,0[Cx)) = Q ifpy^O, 
H'>{B,0{Cx))^nx, 

where 0{Cx) is the sheaf of sections of Cx and tta denotes the irreducible repre- 
sentation of Gr of highest weight A. Then N. Berline and M. Vergne jBV| . |BGV| 
observed that the character of tta, as a function on ga, can be expressed as an inte- 
gral over i3 of a certain naturally defined equivariantly closed form. They proved it 
by applying their famous integral localization formula and matching contributions 
from fixed points with terms of the Weyl character formula. (This is a restatement 
of Kirillov's character formula.) 

M. Kashiwara and W. Schmid [KaSchm] generalize the Borel- Weil-Bott construc- 
tion. Instead of line bundles over the flag variety B they consider GR-equivariant 
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sheaves^ T and, for each integer p G Z, they define representations of Gm in 
Ext^ (JF, OgoTi ) , where Ogon denotes the sheaf of analytic functions on B. In other 
words, M. Kashiwara and W. Schniid prove that these vector spaces possess a natu- 
ral Frechet topology and the resulting representations of Gr are admissible of finite 
length. Hence these representations have characters in the sense of Harish-Chandra. 
Let Q be the character of the virtual representation of Gr 

^(-lfExtP(D6.F,06..(A)), 

V 

where DgJ^ denotes the Verdier dual of JF, Oe"" and A is some twisting parameter 
lying in [}^ - the dual space of the universal Cartan algebra of g^. Here the character 
is a distribution on gR. Then W. Schniid and K. Vilonen jSchV2| prove two 
character formulas for Q. 

The integral character formula expresses 9 as an integral of a certain differential 
form (independent of T) over the characteristic cycle Ch[T) of T . Characteris- 
tic cycles were introduced by M. Kashiwara and their definition can be found in 
jKaSchaj . On the other hand, W. Schmid and K. Vilonen give a geometric way to 
understand characteristic cycles in |SchVl| . A comprehensive treatment of char- 
acteristic cycles can be found in |Schii| . The cycle Ch[T) is a conic Lagrangian 
Borel-Moore homology cycle lying inside the cotangent space T*B. If the sheaf 
J- happens to be perverse, the characteristic cycle of J- equals the characteristic 
cycle of the holonomic I?-module corresponding to T via the Riemann-Hilbert cor- 
respondence. Originally existence of such formula over an unspecified cycle was 
established by W. Rossmann in |R,o| . In the special case when the group Gr is 
compact, the integral character formula reduces to Kirillov's character formula. 

On the other hand, Harish-Chandra showed that the distribution Q is given by 
integration against a certain function on qr. This function Fq is an ^c?(Gr)- 
invariant locally L^-function, its restriction to the set of regular semisimple elements 
0m'' of 0M can be represented by an analytic function. According to Harish-Chandra, 
Fq can be expressed as follows. For a regular semisimple element X e gjj* we 
denote by tc(-'^') C flc the unique (complex) Cartan algebra containing X and by 
'^{X) C we denote the root system of gc with respect to tc(-''^)- Since X is 

regular semisimple, the number of Borel subalgebras {bi,...,b|vi/|} containing X is 
exactly the order of the Weyl group W . Each Borel subalgebra fafe € S containing 
X (and hence containing tc(-''^)) determines a positive root system ^'\jyX) C '^[X) 
consisting of all those roots whose root spaces are not contained in bt,; that is 
contains all negative root spaces relative to (X). Then 



(2) 



fc=l -^^Q! 



a{xy 



where mbfc(X)'s are some integer multiplicities. W. Schmid and K. Vilonen give 
a formula for mb^, (^)'s in terms of local cohomology of T (formula (5.25b) in 
|SchV2| '). Pick another positive root system \1/-(X) C ^iX) such that 

Rea(X)<0 for aU a e *^(X). 



^Strictly speaking, T is not a sheaf on B but rather an element of the "Gr equivariant derived 
category on B with twist (—A — p)" denoted by D(3g(B)_A (see |SchV2| and Remark l2l . 
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(If there are roots in 5* which take purely imaginary values on X, there will be 
different possible choices for ^'-(X).) Let B{X) C Gc be the Borel subgroup 
whose Lie algebra contains ic{X) and the positive root spaces corresponding to 
vI'-(X). The action of B{X) on B has precisely \W\ orbits Oi, . . . ,0\w\- Each 
of these orbits Ok is a locally closed subset of B and contains exactly one of the 
Borel subalgebras {bi, . . . , b|vF|} containing X. We order the Borel subalgebras 
containing X and the orbits of B[X) so that bk is contained in Ok, k = 1,. . . , \ W\. 
Then 

(3) m,,{X) = xinh.i^B^)^) = x((j{i>a-oJ* ° (jo.-6)'lD)6^)) 

(recall that DgJF denotes the Verdier dual of T). These multiplicities are exactly 
the local contributions of points bfe G i3 to the Lefschetz fixed point formula, as gen- 
eralized to sheaf cohomology by M. Goresky and R. MacPherson |GM| . W. Schmid 
and K. Vilonen call (0) combined with the fixed point character formula because 
the set of all Borel subalgebras {bi, . . . , b\w\} containing X can be expressed as the 
set of zeroes of the vector field generated by the infinitesimal action of X on B. In 
the special case when the group Gr is compact, and A + p is an integral weight, all 
the multiplicities mt,j. {X) are equal to each other, say, 

^mb^{X) = • • • = mt,|^^|(X) = K e Z, 

and (0) reduces to k times the Weyl character formula. The fixed point formula 
was conjectured by M. Kashiwara |Kaj . and its proof uses the above-mentioned 
generalization of the Lefschetz fixed point formula to sheaf cohomology jGMj . 

There is a striking relationship between these two character formulas. In the 
compact group case N. Berline and M. Vergne |BV| . |BGV| gave a simple proof of 
Kirillov's character formula using their integral localization formula; they matched 
contributions from zeroes of vector fields with terms of the Weyl character formula. 
However, in the non-compact group case their argument breaks down because their 
localization formula works for compact groups only. Originally W. Schmid and 
K. Vilonen ^chV2 proved these character formulas independently of each other 
using representation theory methods. Thus, besides an important representation- 
theoretical result, they formally established existence of an integral localization 
formula in a very special case. In the announcement |Sch| W. Schmid posed a ques- 
tion: "Can this equivalence of character formulas be seen directly without a detour 
to representation theory, just as in the compact case." In |L1| . |L2| I provide such 
a geometric link, then in |L3I I establish a general integral localization formula for 
non-compact group actions. In turn, this article uses the new localization formula 
for non-compact group actions to give a generalization of the integral character 
formula to representations associated to sheaves on manifolds other than the flag 
manifold of gc- 

We describe the integral character formula for 8 in more detail; its ingredients 
will be used in our character formula H18|l . The character is a distribution on 
^c°^(0k) - the space of smooth compactly supported differential forms on of top 
degree. If <^ G O'°''(0r), then we define its Fourier transform tp g C°°(g^) as in 
PI - PI - p| . and |SchV2| : 

(4) 0{O= [ ^€0M, 
without the customary factor of i = \/"-T in the exponent. 
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For a smooth complex algebraic manifold M we denote by um the canonical 
complex symplectic form on the holomorphic cotangent space T*M . In general, 
when the group Gc acts on a complex manifold M , for X e gC: we denote by Xm 
the vector field on M given by (notice the minus sign) 

(5) {XM-^){x)^^v{e,qy{-eX)x) , (pGC°°(M). 

dc 6—0 

Then the moment map on the holomorphic cotangent space : T'*M —> gj. is 
defined by 

(6) iim{C)--X^~{C,Xm). Xegc, CeT*M. 

When the manifold M is the flag variety B we get the complex algebraic symplectic 
form ais on T*B, the moment map /ig : T*B — > and the vector field Xb for each 
element X G gc- 

We fix a compact real form (i.e. a maximal compact subgroup) ?7r C Gc with 
Lie algebra Ur. We will use Rossmann's map B 3 x ^ & defined in |Ko| and 
Section 8 of 'SchVl' (here we use the notations of |SchVl| and |SchV2| ). Recall 
that the twisting parameter A is an element of the dual of the universal Cartan 
algebra ()c of gc; t)C is not a subalgebra of gcj but is canonically isomorphic to any 
Cartan subalgebra tc C gc equipped with a specified choice of positive root system 
<i>+. For x e we denote by hx C gc the Borel subalgebra corresponding to x. 
Then [)c is canonically isomorphic to the quotient b2;/[ba;, hx], so that hx contains 
all the negative root spaces. Thus we have an exact sequence of vector spaces 

(7) 0^[b,,bj^b, ^f)c^O. 

In general, this sequence does not have a canonical splitting. But once a choice of 
a compact real form [/r C Gc is made, we can split this sequence as follows. Let 
Tk(x) be the stabilizer of x in JJr, it is a maximal torus in [/r, and set 

Im(x) = Lie{TM,{x)) C ur C gc, tc(x) = tR(a;) (g)M C C gc- 

Then icix) is a Cartan subalgebra of gc which lies in b^; and comes equipped with a 

at hx conti 

\)c^U:{.x) ^ bx 

which splits ((T)): 

bx tc{x) © [bx, bx] =i t)c ® [b:,, bx]. 
We also get a splitting of gc: 

gc — tc{x) © root spaces of tc{x)^ ~ [)c © root spaces of tc(2^) 

Taking duals we get a splitting of gj^: 

g^ ~ tc(a;)* © (^^^ root spaces of tc(2;)^ ~ [)J © {^^^ I'oot spaces of tc(a; 

Hence X € i)^ gets identified via this splitting with an element Xx (z Qq. The map 
Xx ■ B ^ is smooth, real algebraic and t/R-equi variant, but in general not Gc- 
or GiR-equi variant, nor complex algebraic. When A is regular, the twisted moment 
map 

Ma =def f-B + Xx : T*B gc 



system of positive roots so that bx contains all the negative root spaces. Hence 
we get a composition of maps 
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is a real algebraic J7]R-equivariant diffeomorphism of T*B onto the coadjoint orbit 
of Gc of any point Xx ^ 9c- 

We will also use a i/R-invariant 2-form t\ on B defined by the formula 

T^{Xx,Yx) ^ Xxi[X,Y]), 

where X^jY^ & T^B are the tangent vectors &i x € B induced hy X,Y e ur via 
differentiation of the t/R-action. It is an important property of A^^ and t\ that 
together they make a (non-homogeneous) differential form on B 

{X, Xx) - ta, X e Qc, 

which depends holomorphically on X (in fact linearly) and which is equivariantly 
closed with respect to [/r (see fBG V) . |GS| or |L3| for the definition of equivariantly 
closed forms). It follows that the forms 

are C/R-equivariantly closed forms too. For a differential form w which is possibly 
non- homogeneous, we denote by ^[fc] its homogeneous component of degree k. Then 
the integral character formula (Theorem 3.8 in ;SchV2 ) says that the value of the 
character 6 at (p E r2*°P(0R) is given by 

(8) g(^) - ro \,,, I (^°MA)-(-^6 + rA)'^ 



1 



{-2TnY JchCF) Vej, ^ [2d] 

X e 0R, C e T*B, 

where ^\ = fJ-B + Xx and d = dime 13. This integral converges because the expression 
1^ o involves the Fourier transform of Lp and decays rapidly along the support of 
Ch{T). 

3. Setup 

Recall that Gc is a connected complex algebraic reductive group which is defined 
over R, Gc acts algebraically on a smooth complex projective variety M . And we 
are primarily interested in representations of a real reductive group Gr C Gc lying 
between the group of real points Gc(IR) and the identity component Gc(M)°. 

We will be using the concepts of P-modules and derived categories; |Bo| and 
|KaScha| are good introductions to these subjects. Let Om denote the structure 
sheaf on M and let Vm denote the sheaf of linear differential operators on M 
with algebraic coefficients; "Dm acts on Om- The definition of a quasi-equivariant 
I?-module can be found in many different sources; for convenience, we copy the 
definition given in 

Let Gc be a connected complex algebraic reductive group acting algebraically on 
a smooth complex algebraic variety M, gc — Lie{Gc)- We write fi for the action 
morphism and tt for the projection map Gc x M —> M: 

/i, TT : Gc X M ^ M, ^{g, x) — gx, iT{g, x) = x g £ Gc, x G M. 

We also consider three maps 

qj : Gc X Gc X M ^ Gc x M, j = 1, 2, 3, 

qi{gi,92,x) = {gi,g2x), q2{gi,g2,x) = {gig2,x), 93(31, ff2, x) = (32, x). 
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Then we have the following identities: 

fioqi=^oq2: {gi, g2, x) i-^ gig2X, 

TT o q2 = TT o qs : {gi, g2, x) i-^ x, 

fj,oqs = TToqi : {gi, g2, x) i-^ g2X. 

Definition 1. We denote by Ogc ^T^m the subalgebra OgcxM ^tt-iOm """^^Af 
ofDccxM ■ A quasi- Gc-equi variant I?M-niodule is a "Dm -"module 9Jl equipped with 
an Oqc ^T^M-linear isomorphism j3 : /i*SUl 7r*9Jl, such that the composition of 
isomorphisms 

q^^i*m ~ ql^i*Tl qlTT*m ~ 93^*971 q*TT*m ~ q^TT*m 

coincides with 

q^H*'m q^Tr*m. 
If (3 is linear even over 'Dq^.^^m, this reduces to the usual definition of a Gc- 
equivariant I?j\/-module. 

Informally speaking, this definition can be interpreted as follows. For each g e 
Gc, denote by iig : AI M the translation by i.e. fig : x t—^ gx. Then /3 
consists of a family of isomorphisms of I?j\/-modules Pg : /i*9Jl fDt, depending 
algebraically on g and multiplicative in the variable g. 

Example 2. Let (E, Ve) be a Gc-equivariant algebraic line bundle over a Gq- 
invariant open algebraic subset U C M with a Gc-invariant algebraic flat connection 
Ve- Take 971 to be the direct image of the sheaf of sections 0(E), under the inclusion 
map U ^ AI . Then Dm acts on 0(E) via the flat connection making 0(E) a quasi- 
Gc-equivariant Vm-module. 

Each element X d qc o.cts on 9Jl in two ways. One way is by inducing the vector 
field —Xm given by 0^ which in turn operates on 271 via the connection. The other 
way is by infinitesimal translation of the sections of the Gc-equivariant line bundle 
E. When these two actions of qc coincide, the T>M-module 971 is Gc-equivariant. 

Let 9Jl be a coherent, quasi-Gc-equivariant I?7\f-module. Recall that, for X G gc, 
Xm denotes the vector field on M given by (0). Following KaSchml we get two 
different actions of gc on Tl. The first action is via qc 3 X i-^ —Xm G ^{'Dm) - 
the global sections of Vm - followed by the 'DM-inodule structure; we denote this 
action by ax>- And the second action is through differentiation of the Gc-action 
when we regard 971 as a Gc-equivariant Oj\/-module; this action is denoted by at. 
We set J = at — a-p , then 7 is a Lie algebra homomorphism 

(9) 7:0c-^Endi,„,(97l). 

This way 971 becomes a (I?M,^(0c))-inodule, where ^^(gc) denotes the universal en- 
veloping algebra of gc- The quasi-Gc-equivariant I?M-inodule 971 is Gc-equivariant 
precisely when 7 = 0. 

We say that the quasi-Gc-equivariant I?M-niodule 971 is Z(gc)-/f'^i^e if some 
ideal of finite codimension T C Z{2c) (the center of L{{gc)) annihilates 971 via the 
7-action. 

We denote by Gh{VJl) the characteristic cycle of 971 which lies in T*M. Pick a 
Borel subalgebra be C flc and define a subset of 

be ={ee0c; Clbc^O}. 
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Then the I?M-niodule 9Jl is called admissible if 

Ch{m)nfijl{b^) c T*M 

is a Lagrangian variety. When DJl is Z(gc)-finite, the variety Ch{Tl) n fi^j{b^) is 
known to be involutive f |KaMF| or jGip. hence the above condition is equivalent 
to 

dimc(C/i(an) n /Xm'(&c )) - dime M. 
Because of Gc-invariance of Cft.(3Jl), if this condition is satisfied for one particular 
Borel subalgebra be C flc, then it is satisfied for all Borel subalgebras of gc, and 
this definition is independent of the choice of the Borel subalgebra be- 

Let iS be a GR-cquivariant constructible sheaf on M. We denote by Oi\ian 
the sheaf of holomorphic functions on M. Then M. Kashiwara and W. Schmid 
|KaSchm| equip the vector spaces 

(10) RHoniP^(aJt(g)5,C'A/a„), p e Z, 

with a natural Frechet topology and prove that the resulting virtual representation 
of Gk 

(11) ^(-l)PRHom?,^^(9Jl®5,OM-) 

p 

is admissible of finite length whenever the I?j\/-module Tl is admissible and Z{qc)- 
finite. In particular, the representation Hll|l has a character 6 in the sense of 
Harish-Chandra. As before, is a distribution on gR. 

In this paragraph we outline M. Kashiwara and W. Schmid's construction of 
topology on the spaces lfTn|l . Oversimplifying and ignoring the Gc- and Gr- actions, 
suppose first that the sheaf S is {ju^M)\^u , where [/ C M is an open semi- 
algebraic GR-invariant subset and Cu is the constant sheaf on U. Furthermore, sup- 
pose that 371 is a locally free quasi-Gc-equivariant I?M-inodulc, i.e. 971 — Dm ®Om 'S 
for some coherent, locally free, Gc-equivariant O^-module ^ on M. They re- 
place the sheaf Om^" with the C°° Dolbeault complex ^^^'\ to which it is quasi- 
isomorphic, and write out isomorphisms of complexes of vector spaces without any 
topology: 

(12) RHomp^, (971 ® S, Om^^ ) ^ RHomp,, {Vm ®Om 'S ® {ju^m)\Cu, nf^'^) 

RHomo,,(5® (jt/^M)!Cc/,r!i"'-^) ^ Rr{U; (r)"' ®o,,.„ 4f'^) 

^r{u; {rr ®om.. 

where (:J*)''" denotes the sheaf of analytic sections of the dual of ^. The complex on 
the right has a natural Frechet topology - the C°° topology for differential forms - 
and continuous Gr action. General 971 and S, have resolutions by locally free quasi- 
Gc-equivariant r>M-inodules (Lemma 4.7 in IKaSchml ) and by sheaves of the type 
iju^M)\'Cu respectively. This, combined with the acyclicity of the right hand side 
of ((T^ . makes it possible to equip the vector spaces (jTU]) with a Frechet topology. 
Then M. Kashiwara and W. Schmid work hard to show that this topology does not 
depend on the choices involved. They do it by proving that the topology on the 
spaces Hl()|> is that of the maximal globalization of their underlying Harish-Chandra 
modules. 
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Recall that B denotes the flag variety of gc- We will establish a formula for this 
character as an integral over a cycle in T*{B x M) under an additional assumption 
that 2Jl has an infinitesimal character, i.e. Z{qc) acts on 9Jl by a character. Even if 
this condition is not satisfied, since 971 is assumed to be Z(0c)-finite, there is a finite 
filtration of 9Jt by quasi-Gc-equivariant I?Af-submodules such that the successive 
quotients have infinitesimal characters, and we can apply this integral character 
formula to each of these quotients separately. 

In the special case when Gc = , the I?A/-module 9Jl is the sheaf of sections of 
a Gc-equivariant line bundle (E,Ve) over Af as in Example 121 with U = M, and 
S is the constant sheaf Cm, the flag variety B consists of just one point and the 
integral character formula will reduce to the classical Riemann-Roch-Hirzcbruch 
formula. 

Remark 1. We do not need M to be projective to establish that is admissible 
of finite length. In fact it is sufficient to assume that M is a smooth quasi-projective 
variety. The compactness of M will be needed for the integral character formula. 

On the other hand, the result of Sumihiro [Suj restated as Proposition 4-6 in 
[KaSchni] together with Theorem 5.12 there show that we can always embed M into 
a smooth projective variety and there is no loss of generality in assuming that M 
is projective. We will illustrate this in Section\^ 

Our derivation of the integral character formula for 6 follows the following 
scheme: 

• Replace the pair (971, S) which lives on M with a new pair (971, S) which 
lives on the flag variety B such that the virtual representation Hll|l stays 
unchanged 

• Write out the flxed point character formula for 9 in terms of (971, S) and 
zeroes on B; 

• We want to prove that the integral (jHl represents 9; flrst we apply the 
integral localization formula (LS, to ©; 

• Combine similar terms in the result of the previous step and match them 
with the terms in the fixed point character formula for 9 obtained in the 
earlier step. 

4. Statement of the main result 

In the previous section we assumed that 97t was a quasi-Gc-equivariant coherent 
I?A/-module on M which was admissible and had an infinitesimal character. We 
will follow |KaSchmj and index characters of 2{2c) by linear functionals A on the 
universal Cartan without the customary shift by p (half sum of the positive roots); 
in other words, xa : -2(gc) ^ C denotes the character by which Z{gc) acts on 
the Verma module with highest weight A. Then x\ = ^-"^^ only if A + p is 

conjugate to n + p under the action of the Weyl group W. 

Remark 2. In this article we use results from KaSchm and |SchV2j . Unfortu- 
nately these two sources use different conventions for labeling characters and twists. 
We follow the conventions of (KaSchm) explained above. On the other hand |SchV2| 
shift these notations by p and they define, for instance, the "G^-equivariant de- 
rived category on B with twist (A — p) " denoted by T)g^{B)\ which becomes the 
bounded Gs,-equivariant derived category in the usual "untwisted" sense precisely 
when X = p. Similarly, C'b(A) in [SchV2j denotes the twisted sheaf of holomorphic 



10 



MATVEI LIBINE 



functions with twist (A — p) and which becomes the sheaf of ordinary holomorphic 
functions precisely when X — p. 

We assume that 971 is an object in Mod^°^'g^lQ^_g^(I?Af ) - the category of quasi- 
Gc-equivariant coherent I?A/-inodules on A4 with infinitesimal character xa-p- 

The Kashiwara-Schmid construction of Frechet topology on the spaces (|1(J|I was 
carried out on the level of derived categories. This means that instead of a sheaf 
S we have an element S S Dq^ k-c(^m) ~ the bounded derived category of Gr- 
equivariant R-constructible sheaves, and we have a pairing 

{m,s) ^ RRom'°i {m®s,o Ma.) , 

where D''{Tgr) denotes the derived category of Gn-representations defined in §3 
of |KaSchm| . The category D^{Tg») is built on complexes of topological vector 
spaces with continuous Gjn-actions (C,dc\ the differential maps : 
are required to be continuous and GR-equivariant. A complex (G, dc) is exact if, 
for all n, 

dj^ : G" ^kerd^+^ 

is onto and is an open map relative to the subspace topology on ker d^^ . A complex 
{C,dc) becomes zero in D^{!Fgu) precisely when it is exact. 

Because we will apply results of A. Beilinson and J. Bernstein on equivalences 
of categories |BB| . we will also assume that A is integrally dominant, i.e. 

(a, A) ^ Z<o, for every positive coroot a. 

This can always be achieved by replacing A with an appropriate W-translate. 

We denote by ModGc-eg(2^e, A-p) the category of modules over the sheaf of 
twisted differential operators A-p on the flag variety B. The sheaf Vb, \~p is 
defined in BB , but we follow the twisting conventions of [KaSchrn] as explained in 
Remark|21 so that Vs. o = 'Db - the sheaf of differential operators on B without any 
twist. We form a product space B x M with diagonal Gc-action and consider the 
sheaf of twisted differential operators Vbxm.x-p ^def T^B.x-p ^ T^m, the twisting 
is confined to the factor B. We also denote by ModGc-eq{T^8xM,\-p) the category 
of Gc-equivariant PgxM, A-p-modules on B x M. Let p and q be the projection 
maps 

B BxM 

M, 

and let p and q be the induced projections on the cotangent bundles: 

T*B ^ — T*{B X M) 

I' 

T*M. 

M. Kashiwara and W. Schmid "KaSchmj use results of A. Beilinson and J. Bern- 
stein on equivalence of categories ^BB, to prove that the pair (9Jl, S) on M can be 
replaced with a Vs, p_A-module and a complex of sheaves on B. We will make this 
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statement precise. First of all, they show that there exists a coherent holonomic 
module £ G Modoc-egiT^BxAL \~p) such that 

g*£ = m and R''q^£ = if /c 7^ 0, 

namely one can take 

£ = I?B,A-p K$H/7BxM(flc)(2?e,A-p KOT), 

where 7sxm(0c)(2'z3, A-p 9Jl) denotes the image in I?e, A-p Kl 9}t of the module 
flciX" (I'b, A-pK19}t) under the map jbxm given by the equation jnj with the ambient 
manifold B x M. 

Then we apply the twisted deRham functor to obtain a complex of sheaves 
C = DRbxm{^) =def RnomT>^^^„^_^{OB{\ - p) H Oa/, -C), 

which is an element of the bounded Gc-equivariant, C-constructible derived cat- 
egory with twist (p — A) along the S-factor denoted p-\ c-d'^Bxht)- Here 
Cb(A — p) is a twisted sheaf of holomorphic functions on B, with twist (A — p) (so 
that Ob(0) is just the sheaf of functions on B with no twist at all). 

Let d = dime and let (ilg)^^ ~ Oisi^p) denote the reciprocal of the canonical 
sheaf. Combining the equations (6.7), (7.14) and (7.15) from |KaSchni) we obtain 

(13) RHom^J'^(9Jl(g)5,OAf-) 

~ RHom^°^^ {Vb, p-x ®Os (f^e)"' ® Rp*iC®q-'S),OB^'^ (p- A)) [d-2 dime M] 
~ RRom'°l _^_p{VB,-x-p ® Rp.{C ® q-^S),OB^^ (-A - p)) [d - 2 dime M] 
~ RHom(i?p* (£ (g) q'^S),OB-^ (-A - p)) [d-2 dime M] 

as elements of the derived category of GR-representations D^{Tgk)- Here we view 
Rp^,{C®q~^S) as an object in f^j^ _;^_p r_c(Cb), which makes sense because 2p is 
an integral weight, and this implies the existence of a canonical isomorphism (6.8) 
in |KaSchin| 

D^g^p-XM^ci'^B) - ^Ge -A-p,R-c(Cb)- 

That is, the pair (9Jl, S) on the variety M is replaced by a pair 

(14) {VB,-x-p,Rp*{C®q-^S)) 

on the flag variety S, with an additional twist by (—A — p). 

Our starting point is the integral character formula (|SJ) proved by W. Schmid and 
K. Vilonen in |SchV2| . We apply it to the right hand side of l|13|l . We fix a compact 
real form C/r C Gc. Recall that pB ■ T*B the moment map defined by 

(inj, CFB is the canonical complex algebraic holomorphic symplectic form on T*B, 
\x '■ B q}i- is the J7R-equivariant Rossmann's map, and ta is a certain J/R-invariant 
2-form on B. The character of the virtual representation (|ll|l is a distribution on 
^c°^(0k) - the space of smooth compactly supported difl'erential forms on 0h of top 
degree. For an element € ri*°^(gR), its Fourier transform ifi G C°°{g^) is defined 
by Q without the customary factor of i = %/— T in the exponent. Then the integral 
character formula says that the character 6 of the virtual representation of Gr, 
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as a distribution on QI°p{qr), is 



(27ri)'' Jc/i(flp.(£®<j-i5))" ^ [2d] 

X e 0K, C e T*B. 

Here = IJ-b — ^x, d — dimc^B, a : T*B T*B is the antipodal map C, i— 
— C, and Ch{Rp^,{£ (g) q~^S))'^ denotes the image under this antipodal map of the 
characteristic cycle of Rp^,{C (E) q~^S) (which is a cycle in T*B). 

Remark 3. If Z is a complex manifold and Z"^ is the underlying real analytic 
manifold, then the holomorphic symplectic form az is defined on the holomorphic 
cotangent bundle T*Z , while the characteristic cycles of constructible sheaves on 
Z lie in the real cotangent bundle T*{Z^). Hence we need to identify T* Z with 
T*{Z^). There are at least two different but equally natural ways of doing this, we 
use the convention (11.1.2) of [KaSchaj . Chapter XI; the same convention is used 
in |L1| , jL2| , |L3| and |SchV2| . Under this convention, if az,R is the canonical real 
symplectic form on T*Z^ and az is the canonical complex holomorphic symplectic 
form on T*Z, then az,m gets identified with 2Keaz. 

Another important ingredient is a generalization of the Hopf index theorem 
stated as Corollary 9.5.2 in |KaScha| . Let T be a constructible sheaf on M or 
an element of the derived category D^_^{Cm)i and let x{M,T) denote the Euler 
characteristic of M with respect to T. Then 

(16) x(M,r) = #([M]nC/i(T)), 

where [M] denotes the fundamental cycle of M. Alternatively one can apply the 
equation (5.30) from jSchii| . Let Thom^* m denote the Thom form of the cotangent 
bundle T*M M . That is Thomy* m is a closed differential form on T*M of degree 
2 dime M which decays rapidly along the fiber (or even compactly supported along 
the fiber) and such that 

( Thomr-M = (27r)'i™^^^ Vx G M. 
Jt* m 

We regard M as a submanifold of T*M via the zero section inclusion. Then 
the restriction of the Thom form to M is the Euler form of M . Since the form 
(27r)-dimcM xhomT-A/ is Poincare dual to the homology class of [M] in T*M, the 
Hopf index theorem can be rewritten as 

(17) x(M,r) = #([M] n Ch{T)) = (2^)-d"ncM j ThomT-A/, 

JCh{T) 

which is a generalization of the Gauss-Bonnet theorem. 

Recall that [/r is a compact real form of Gc. The form Thom^M may be chosen 
to be C/R-invariant. If, in addition, the cotangent bundle T*M has a spin structure, 
then V. Mathai and D. Quillen showed in "MQj (see also Section 7.7 in jBGV|) that 
ThomT* M can be realized as the top degree part of a J/K-equivariantly closed form 
on T* M in a canonical way. 
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The last essential ingredient is the deformation argument for integrals of equiv- 
ariantly closed forms from |L8| . This argument requires that any maximal complex 
torus Tc C Gc acts on M with isolated fixed points. (Since M is compact, there 
will be only finitely many of those.) If this condition is satisfied for one particular 
torus, then it is satisfied for all tori because all maximal complex tori are conjugate 
by elements of Gc- (This condition is satisfied in all application we have in mind.) 

We will combine the integral character formula 1)15(1 and the Gauss-Bonnet for- 
mula H17|l to get a formula for the character of the virtual representation (|13|) as 
an integral over a cycle in T*{B x M). Set 

S = C(g)q-^S and A = Ch{S)'' = Ch{BBxM{§)) cT*{Bx M). 

Here a : T*{B x M) T*{B x M) is the antipodal map C ^ -C, and Ch{SY 
denotes the image under this antipodal map of the characteristic cycle of S] the 
operator D^xm is the Verdier duality operator (see below for its properties). 

Theorem 1. Suppose that any maximal complex torus Tc C Gc acts on M with 
isolated fixed points. Then the value of the character 9 of the virtual representation 

on^e nl^pieu) is 

(18) d{^) = (2^,)I+«rf, /^P*((^ ° A^-a) • (^8 + r^r) A q* ThoniT-M 

= '1,^ I i I P*e<^'''''(^'-^->+"^+"^ ArThomT.MA¥>(X)) 
where fi_\ = fJ-B ^ -^x, d ^ dime ^ o,''^^ n = dime 

Remark 4. Suppose that the group Gc = C^, the T>m -module OJt is the sheaf of 
sections of a Gc-equivariant line bundle (E, Ve) over M as in Example\^ and S is 
the constant sheaf C m ■ Then the flag variety B is just one point, the cycle A = [M] , 
and because of the Riemann-Roch relationship (8.4) in |MQ| (or Theorem 7.44 ^n 
|BGV) ) the above integral character formula will reduce to the classical Riemann- 
Roch- Hirzebruch formula. There is no curvature of the line bundle (E, Ve) present 
in the character formula because we assume that the connection Ve is flat so 
that 0(E) is a quasi-Gc-equivariant T> m -module. 



5. Proof of Theorem ^ 

We start our proof by applying the fixed point character formula ||2Jl combined 
with (O due to W. Schmid and K. Vilonen |SchV2| to the right hand side of 
((13|l . Thus the character of the virtual representation ((13|l is given by integration 
against a function Fg on 2r' 

(19) ei^)^ I Fe^, G f^c°^(0K)- 

This function Fg is an Afi(GK)-invariant, locally function on whose restriction 
to the set of regular semisimple elements can be represented by a real analytic 
function. The value of this analytic function at X G is determined by zeroes of 
the vector field Xb on the flag variety B as follows. 

Recall that tc(-'^^) C gc is the unique Cartan subalgebra containing X e . Let 
^'(X) C tc{X) be the root system of gc with respect to tc(-'^^) and pick a positive 
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root system ^/-{X) C ^' such that 

Rea{X)<0 for all a G 

Let B{X) C Gc be the Borel subgroup whose Lie algebra contains tc(-^) and the 
positive root spaces corresponding to '^-{X). The action oi B{X) on B has exactly 
the same number of orbits Oi, . . . , O^^i as the order of the Weyl group W of Gc- 
Each of these orbits Ok contains exactly one zero of the vector field Xs, and we 
order the zeroes {bi, . . . , b^^y^} of Xg so that bk is contained in Ok, k — 1, . . . , \W\. 
The set of zeroes {bi,...,b|vi/|}is precisely the set of Borel subalgebras containing 
ic{X). Let 'i''^^{X) C ^'(-Y) be the positive root system such that bfe contains all 
the negative root spaces, k = 1, . . . ,\W\. Then the fixed point character formula 
|SchV2| says that the function Fg which appeared in the equation H19|l is 



\w\ 

(20) FeiX) ^ {^irY.'^^^^^^u^ 

k=i llae*+ 



where mt,^(X)'s are integer multiplicities given by the formula 

(21) m,,{X) = x{nh,{Rp*{S)),,) = x((j{f,a-oJ* ° {jo,^b)\Rp*{S))). 

Let Dg and DgxM denote the Verdier duality operators on B and B y. M respec- 
tively: 

^B-- D'a.^_^^p,R-c{^B) ^g«,a+p,k-c(Cb); 



(22) ObxM ■■ ^Ge -A-p,R-c(CBxAf) DQ^,,X+p,R-ci^Bx m) ■ 

The effect of the Verdier duality operator on the characteristic cycle of an M- 
constructible sheaf T (or an element of D^_^{Cz)) on any smooth quasi-projective 
variety Z is described by 

Ch{Bz{T)) = CT,(r)^ 

where a : T*Z T*Z is the antipodal map ( i-^ and Ch{TY denotes the image 
under this antipodal map of the characteristic cycle of T . 

For a regular semisimple element X g q}? we denote by Tc{X) = exp(tc(X)) C 
Gc the maximal complex torus corresponding to the unique Cartan subalgebra 
containing X. If a; G M is a point fixed by Tc(X); then Tc{X) acts linearly on 
the tangent space T^M. We denote by A(X) c tc(-'^)* the set of weights of tc(-'^) 
which are either roots of gc or occur in the tangent space T^M of some point x € M 
fixed by Tc{X). 

Let g^ denote the set of strongly regular semisimple elements X G q!^ which 
satisfy the following additional properties. If tR(X) C 0e and ic{X) C gc are the 
unique Cartan subalgebras in gg and gc respectively containing X , then: 

(1) The set of zeroes of the vector field Xm is exactly the set of points in M 
fixed by the complex torus Tc{X) — exp(tc(X)) c Gc; 

(2) P{X) + for all /3 G A(X) C tc(X)*; 

(3) For each /3 G A(X), we have either 



(23) 



Re(/3)|t,(x) = or Re(/3(X)) ^ 0. 
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Clearly, is an open subset of 0r; since M is compact and A(X) is finite, the 
complement of q'^ in gR has measure zero. 

From now on we will assume that the element X g gu is not only regular semisim- 
ple, but also lies in gjj. Then applying the integral localization formula of Jj3, to 
the integral character formula H15() . with the global formula for the multiplicities, 
we can rewrite the formula H21|l as 

(24) 

m,,{X) = x{B, {Bb{Rp.{S)))^J = x{B, {jo,^b)< ° (jo.^s)*(D6(i?P*(5)))). 
Applying Proposition 2.5.11 from jKaSchaj to the Cartesian square 

OkX AI Bx M 

I I 
Ok ^ B 

and using that the projection map p is proper we obtain: 
(25) m,,{X) = x{B, {jo,^b)< o {jo.^b)* {RP'X^b.m {§)))) 

= X{B, {jOk^B)i. ° RP\ o {iOkXM^BxM)* {Obxm{S))) 
= x{B,Rp\ o (jOfcXM-^BxM)! o (jOfcXM-^BxM)*(Di3xM(5))) 
^X{BX M, (jOkXM^BxM)<. o UOkXM^BxMTi^BxMiS))) 

^X{BXM, (DexM(5))o,xM). 

Next we will compare this result with the result of application of the integral 
localization formula of |L3j to the integral H18|l . Let 

M= 11 

be the Bialynicki-Birula decomposition [Bi] of M into attracting sets (relative to 
X), as described in To obtain this decomposition we pick any X' e tR(X)n0K 
in the same connected component of tR(X) n gjj and such that 

Re/3(X) > Re/3(X') > and Re/3(X) < ^ Rc/3(X') < 

for all /3 £ A(X), and the complex 1-dimensional subspace {tX'; t G C} C gc is 
the Lie algebra of a closed algebraic subgroup C^{X') C Gc isomorphic to C^. 
We fix an embedding of C^(X') ~ into C so that the tangent map sends 
X' £ Te{C^ {X')) into an element with nonnegative real part. This embedding 
allows us to take limits as z g (X') approaches to zero, and for each x £ M with 
Xm{x) = we set 

Ox = {y e M; lim z"^ ■ y ^ x}. 

The sets Ox are smooth locally closed algebraic subvarieties of M. 
Then 

{x£Bx M; Xbxm{x) = 0} 

= {x£B; Xis{x) = 0} X {a; e M; Xm{x) = 0} 

= {bi, ...,b\w\}^ {x£ M; Xm{x) = 0} 
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is the set of zeroes of the vector field Xbxm, and 

k^l,...,\W\ 
{x e M; Xm{x) = 0} 

is the Bialynicki-Birula decomposition of S x A/ into attracting sets (relative to 
X). Applying the integral localization formula to the integral (|18|l yields 

I. (I A r Thom..M ^,iX)) ^^^^^^^^ ^ I Fe, 

for any £ with supp^j C gg, where the function Fg is 

(26) Fe{X)^{-lYY. E ^nkAXh 



Bh 



fc=i {x^M- Xm{x)=o} nae*+^ (X) "(^) ' 

and 

muAX) = x{B X M, (DexAf(5))o,x6j 

= X(S X Af, (jo,x6.^i?xAf)! ° (io.x6.^i?xAf)*(DexM(5))). 
We want to show that F0{X) = Ffl(X). We have: 

(DexM(5))Q^xo^) 

{xeM- Xm(x)=Q} {xeM- XM{x)=a} 

= X{B X A/, (DexAf(5))o,xA/) = mb,(X). 

Combining this with the equations H2U|I and (|26|l we obtain 

\w\ -{X\i ) 
Fe{X) = [-lYY.mx,—l 1^1^ = Fe{X), 

which proves (|18|l at least when supp((/3) C gjj. Since the function Fg is known to 
be a character of some representation (namely (C3), it is locally and the main 
result of |E3] now proves (O for aU ip e r2*°P(0R). 

6. An example on the enhanced flag variety 

The purpose of this section is to show how Theorem ^ can be applied to smooth 
quasi-projective varieties which are not necessarily projective. We let our group 
Gr be GL(7V,M) C GL{N,C) = Gc, for some A^ e N; we denote by Be the group 
of all invertible lower-triangular matrices and by Ac C Be the group of lower- 
triangular unipotent matrices, we set Br = BqI^G^ and Ar — A^^I^Gr. We will be 
interested in the complex homogeneous space = Gc/Nq and its real submanifold 
Zr = Gr/Ar C Zc- The space Zc can be regarded as a fiber bundle over the flag 
variety B = Gc/Bc with fibers isomorphic to He =def Bc/Nc — (C^)^. It is 
easy to see that these fibers can be identified with each other in a canonical way. 
This allows us to think of Zc as a trivial principal i?c-bundle which is also a Gc- 
equivariant fiber bundle, and the actions of He and Gc commute. W. Schmid and 
K. Vilonen call Zc the enhanced flag variety of gc |SchV2j . 
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Let S' = (izK^Zc)*(CziJ denote the sheaf on Zc which is the direct image of 
the constant sheaf on Zr. Fix a A G ()J and denote by kerxA-p C Z{2c) the kernel 
of the character XA-p : 2{gc) C, and take a P^c-niodule 



Notice that by construction Z{gc) acts on Tl' by character xa-p- We form a virtual 
representation of Gr 



The composition factors of this representation are known to be the principal series 
representations of Gk which have the same character on gR. We will obtain an in- 
tegral formula for the character of this representation. Of course, the homogeneous 
space Zc is not compact, and in order to apply Theorem ^ we need to compactify 
Zc first. 

Recall that Zc is a principal {He, Gc)-equivariant fiber bundle with commuting 
He- and Gc-actions. Hence we can embed He =def Bc/Nc ~ (C)^ into the 
vector space and form a Gc-equivariant vector bundle Zc ~ x B with an 
embedding Zc ^ Zc of Gc-equivariant fiber bundles over B. Next we form a line 
bundle Lq — C x B over B with trivial Gc-action on the first factor and form a 
projectivization M =def TP{Zc ® -^^o)- Then M is a smooth complex projective 
variety, Gc acts on M algebraically, and AI contains Zc as a dense open subset. 
Define 



= X'A//7M(kerxA-p)2?M and S = {jzc'-^M)\S' = {jz^--,M)\{Cz^). 



For convenience we restate Theorem 5.12 of [KaSchm] : 
Theorem 2. Let f : X ^ Y be a Gc-equivariant morphism between complex 
algebraic, quasi-projective Gc-manifolds X , Y . If f is projective, there exists an 
isomorphism 



functorially in T ^ R-c('^>') '^"'^ ^ ^ ^Gc cohi'^x) (the bounded derived 
category of Modgl^^,i^a^_^g{Vx)). If f is smooth, 

RHom^°^(L/*a7l®r,Ox"") ^ RHom^°P (2n(^ i?(/a„).r, Oya.)[-2 dimX/F], 
functorially m M & D'^.^ ^^jVy) and T £ -Dgr.r-cC^^)- 



Applying the second part of this theorem to the open inclusion Zc ^ M we can 
rewrite our virtual representation (|27|l as 



RHom^^^ {m' (g> S' , Oz"" ) - RHom^^ ® 5, Cm- ) ■ 
The 2?A/-module 9Jt is Z(gc)-finite and hes in ^od'^'^'^Zcc-eqi'^M) essentially by 



construction. It follows from the Bruhat decomposition of Gc that Be acts on 
M with finitely many orbits; this implies that VJl is admissible. It is easy to see 
that each maximal torus Tc acts on M with finitely many fixed points. Therefore 
Theorem Q applies here and we obtain an integral formula for the character of the 
virtual representation 1)27(1 . This is probably the most complicated formula for the 
principal series character there is. 



Tl' = X'zc/72c(kerxA-p)2?Zc- 



(27) 



RHom^°^^(ajl'®5',Ozg"). 
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